1. Field of the Invention
The present invention relates generally to video image processing methods and, in an embodiment described herein, more particularly provides a method of stabilizing and registering video images.
2. Description of Related Art
Techniques presently exist for stabilizing video images. These techniques typically function to reduce or eliminate image translation (i.e., displacement) horizontally and vertically in a video sequence. In general, these techniques are very limited in effectiveness, since they are not able to compensate for image rotation or dilation. In addition, these techniques are sensitive to the effects of parallax in which objects in the foreground and background are moving at different rates and/or directions. Furthermore, these techniques are typically able to determine image motion only to the nearest pixel.
Video image stabilization and other image enhancing techniques are also described in the following U.S. published applications: 2002/0064382 2003/0090593 2003/0099410; and U.S. Pat. Nos. 5,784,175 5,453,800 5,327,232 5,210,605 4,924,306 5,815,670 5,742,710 5,734,737 5,686,973 5,535,288 5,528,703 5,778,100 5,748,784 5,748,761 5,745,605 5,737,447 5,734,753 5,729,302 5,703,966 5,684,898 5,581,308 5,555,033 5,488,675 5,488,674 5,473,364 5,325,449 5,259,040 5,067,014 4,797,942 4,675,532 4,937,666 4,979,738 5,144,423 5,263,135 5,276,513 5,278,915 5,321,748 5,518,497 5,534,925 5,566,674 5,627,915 5,629,988 5,635,994 5,657,402 5,717,793 5,909,657 5,920,657 5,963,675 6,037,988 6,173,089 6,571,021 6,640,018 6,373,970 6,650,792 5,943,450 5,204,944 5,050,225 4,908,874 4,893,258 4,759,076 4,672,680 6,459,822 and 6,560,375.
The last two of these (U.S. Pat. Nos. 6,459,822 and 6,560,375), having the present inventor as a coinventor thereof, provide an advanced video image stabilization and registration technique which is very accurate and is capable of compensating for image rotation and dilation, and is capable of compensating for the effects of parallax. Unfortunately, however, this technique does not compensate for other forms of distortion, including different magnifications in different directions (as seen, for example, when an object is rotated toward or away from the camera and thus foreshortened in one direction) and shearing of the image (as seen in more complex object motion). This technique also uses prior knowledge of the shape (width-to-height ratio) of the image elements (pixels) to both determine the changes in the image using its limited image transformation, and to then correct for those changes.
Therefore, it can be seen that it would be quite desirable to provide an improved video image stabilization and enhancement technique which can compensate for additional forms of image distortion, and which does not require advance knowledge of a pixel width-to-height ratio of the image. It is accordingly among the objects of the present invention to provide such a technique.